The invention relates generally to video image processing and more particularly to apparatus, system and a method for obtaining and processing information about the quantity and the direction of movement of people in crowded areas using video image processing.
Monitoring the movement of people entering and leaving major public areas, such as malls, shopping areas, chain-stores, casinos, airports, bus and train stations and sport facilities provides significant and valuable information to the management of these areas.
For example, information regarding the number of entrances and exits of people to and from a particular doorway of a specific store, within a period of time, enables the management to evaluate the store efficiency, and obtain a correlation between the number of visitors and sales. Furthermore, it is possible by checking the number of visitors against employees, to check the employee/customer ratio to determine whether the staff are overloaded and analyze the customer service being obtained vis-a-vis sales.
During the last few years, video image processing systems and visual image processing system have been developed in order to provide information about the movement of people. Methods and apparatus using video based systems for obtaining information about the movement of people are known. For example, WO Application No: 98/08208, assigned to Footfall Ltd., entitled xe2x80x9cVideo Imaging Systemsxe2x80x9d, describes video image sampling at a low resolution and with a homogenous scattering of sample points. The low resolution and sampling scattering saves computer resources. However, since the samples are taken at low resolution with a homogenous scattering of the sample points, the correlation level between one sample and another of the same area is low and there is a danger of losing positive identification of figures due to the low resolution. This system, which uses the resources of a personal computer (PC), and thus this PC is not capable of performing a high-resolution process of the images, which are constantly being received from several end units.
WO Application No: 99/04378, assigned to Footfall Ltd., entitled xe2x80x9cVideo Imaging Systemsxe2x80x9d, describes an improvement of the earlier application (WO 98/08208), described hereinabove, utilizing infra red (IR) light to illuminate the moving figures and recording the image with an IR sensitive video camera. The IR camera and illumination solves the problem of shadowing of their previous patent which is critical in case of low resolution of the video image. Low resolution sampling to distinguish the moving figures from the background and identify them as people is sensitive to shadowing.
However, the addition of IR light resources and IR sensitive video means involve extra costs, making the system more cumbersome and requiring wiring and suitable power sources wherever the units are installed. Usually, these installations are installed in the ceiling, which may, for example, be concrete, thus requiring specialized suspension devices.
U.S. Pat. No. 5,973,732 to Guthrie, entitled xe2x80x9cObject Tracking System For Monitoring a Controlled Spacexe2x80x9d, describes video image processing which initially uses low resolution of xe2x80x9csuper pixelsxe2x80x9d, that is xe2x80x9ccellsxe2x80x9d or xe2x80x9cblocksxe2x80x9d of pixels, instead of a processing at single pixel level. This is done in order to reduce the analysis necessary in order to identify and preliminary locate the moving figures within the area of interest. However, by decreasing the sampling resolution to fixed xe2x80x9ccellsxe2x80x9d or xe2x80x9cblocksxe2x80x9d as mentioned, the system knowingly suffers from an information loss. Thus, since the processing is made according to a net of xe2x80x9ccellsxe2x80x9d or xe2x80x9cblocksxe2x80x9d having a fixed size, there is a danger that an object, which passes on the borders of two xe2x80x9cblocksxe2x80x9d, cannot be distinguished from the fixed background, and could not be identified as a figure.
U.S. Pat. No. 5,465,115 assigned to RCT Systems Inc., entitled xe2x80x9cVideo Traffic Monitor for Retail Establishments and the likexe2x80x9d, describes a video detection process of passing figures as such by positioning an imaginary strip of xe2x80x9cgatesxe2x80x9d or in other words or a line of imaginary static xe2x80x9cturnstilesxe2x80x9d on the video image and identifies the object as a figure when it fills the xe2x80x9cgatexe2x80x9d or passes through the xe2x80x9cturnstile.
The system does not actively search for and track figures over the whole videoed area, but identifies the entry and exit of a figure into and from a the imaginary strip of xe2x80x9cgatesxe2x80x9d, in order to reduce computer resources. This system is inflexiblexe2x80x94entry and exit of static to and from the strip could be considered as figures filling (or emptying) the gates and therefore the system requires re-initialization.
Furthermore, the prior art systems do not integrate data collection into an overall system for client commercial purposes. For example a potential client of a system of this kind, could be a manager of an international chain of stores, who needs a daily report regarding the number of people entering and exiting each of his stores the day before. Modern communication systems which enable information to be gathered and transmitted by telephone, modem, cellular communication and Internet, for example, are well known in the art. However, prior art imaging systems, and visual imaging systems which supply information about the movement of people, do not deal with the combination of such systems with on-line communication systems.
Therefore, in view of the limitations of systems based on the prior art, there is need of a video-based system and method for obtaining information about people, whose end unit is integral, easily installable and standalone, to the greatest possible extent, in terms of integral independent capabilities for picture processing and data accumulation therein. Limitatons on processing resources will not make it necessary to forgo reliability, and it will be possible to integrate the end units in modern communications systems (e.g.xe2x80x94cellular communications, the Internet and combinations of the two), such that data can be transferred to the customer easily, without his having to invest in additional communications and data processing systems.
One object of the present invention is to enable easy and convenient installation of the end unit in a ceiling of any type (ordinary or acoustic). The structure of an end unit according to the invention is adapted for installation in a variety of ceilings, and no other infrastructure is required beyond the electricity supply. An end unit according to the invention is thus adapted to the level of a field technician, who will not have to do complicated assembly work or deal with auxiliary installations in the event that the ceiling is of a different type than what he expected.
A further object of the invention is operational independence. The invention does not require dedicated lighting means extrinsic to the end unit, or infrared photography. An end unit according to the invention relies in its operation on the ordinary lighting which is present in any case in the photographed area, and adjusts itself to this lighting. The problem of shadows and changes that occur in the lighting conditions is solved in the end unit without need of additional, dedicated lighting means.
A further object of the invention is to enable immediate processing, which is performed in a standalone processor integrally installed in the end unit itself. The end unit processes the video image produced by an internal camera installed in the end unit, and even enables parallel reception and processing of two video images arriving simultaneously from internal cameras installed in two separate end units. The standalone processor installed in the end unit itself enables independence from distant external processing units, and the ability of a processor in one end unit to process two images simultaneously makes it possible to increase the area in respect of which the data processing is requested. For examplexe2x80x94for a wide opening in a shopping mall having a relatively low ceiling necessitating two cameras for xe2x80x9ccoveringxe2x80x9d the opening, the invention enables the use of only two end units, one of which is even xe2x80x9cdegeneratexe2x80x9dxe2x80x94i.e. does not contain a processor (and is therefore inexpensive).
A further object of the invention is to enable processing as stated, while choosing specific areas of interest out of the general view seen in the video picture, with the possibility for selection and immediate adjustment of the size and position of the areas. An end unit according to the invention enables the technician installing it to initialize the unit such that out of the general view in the photographed picture, the data processing and production are performed only on specific areas designated on the view as areas of interest (e.g.xe2x80x94elevator openings, staircase, where there is human traffic), and not on areas of no interest (e.g.xe2x80x94static display cabinets).
A further object of the invention is to enable processing as stated at a high resolution, in the critical initial stage of differentiating objects from the photographed background, and identification of the objects as images passing through the areas of interest in the photographed area. The invention does not xe2x80x9csacrificexe2x80x9d information that could represent images, and performs a preliminary identification at a high level, while reducing the likelihood of information loss and error.
A further object of the invention is to enable accumulation of data from the information processed in the end unit, i.e.xe2x80x94the quantities of people and the directions of their movementxe2x80x94in an integral memory installed in the end unit itself, and retrieval of the data from the memory and the transfer thereof from time to time over distances, based on a command sent from a distance. An end unit according to the invention does not rely on external memory components.
A yet further object of the invention is the transfer of data that are retrieved from a distance from the end unit, in a secured manner, via a variety of communication media.
In another preferred configuration of the inventionxe2x80x94the sending of data is done by GSM communications (in addition to or as a substitute for other communications media, such asxe2x80x94cables, RF transmission, telephone line).
A further object of the invention, in another preferred configuration thereof, is to enable the concentration of the retrieved data in a central site on the Web, allowing access to the data only to authorized customers (customer access is limited to data in respect of which the authorization was granted). The central site is able to provide data analysis and processing services for commercial purposes, thereby saving on the costs entailed in the development of independent computerization and data processing capabilities.
In other words, the invention offers a service to subscription customersxe2x80x94a data xe2x80x9cbankxe2x80x9d of sorts whose customers, by xe2x80x9csurfingxe2x80x9d on the Web to the central site, are able to enter the secured xe2x80x9cvault roomxe2x80x9d and obtain their informationxe2x80x94to receive reports and processed statistical presentations based on people entry and departure data as retrieved from the end units according to the invention that were installed in the ceilings ofxe2x80x94for examplexe2x80x94stores under their ownership distributed around the world.
There is thus provided, in accordance with a preferred embodiment of the invention, a method for detecting and counting persons traversing an area being monitored. The method includes the steps of:
initialization of at least one end unit forming part of a video imaging system, the end unit having at least one camera installed therein, the camera producing images within the field of view of the camera of at least part of the area being monitored, the end unit includes at least a non-volatile memory unit and a working memory unit, the non-volatile data memory unit includes a plurality of counters;
digitizing the images and storing the digitized images in at least one of the non-volatile memory unit and a working memory unit;
detecting objects of potential persons from the digitized images;
comparing the digitized images of objects detected in the area being monitored with digitized images stored in the working memory unit to determine whether the detected object is a new figure that has entered the area being monitored or whether the detected object is a known figure, that has remained within the area being monitored and to determine that a figure which was not detected has left the area being monitored; and
incrementing at least one of the plurality of counters with an indication of the number of persons that have passed through the area being monitored.
Furthermore, in accordance with a preferred embodiment of the invention, the method the step of updating further includes at least one of the steps of:
incrementing one of the plurality of counters with an indication of the number of persons that have passed through the area being monitored and have left the areas in a pre-determined direction; and
incrementing one of the plurality of counters with an indication of the number of persons that have passed through the area being monitored and have left the areas in one of at least two directions.
There is also provided, in accordance with a preferred embodiment of the invention, a video based system for detecting and counting persons traversing an area being monitored. The system includes at least one end unit a device for remote retrieval of the data stored in the non-volatile memory unit, a device for processing the data and generating reports therefrom, and a device for displaying the reports.
The end unit includes at least one CCD camera, an A/D converter for converting analog information from the camera to digital information, a digital switching unit connected to the A/D converter, a video processor connected to the digital switching unit, for receiving the digital video information from the digital switching unit; and a non-volatile memory unit and a program working memory unit coupled through an address and data bus to the video processor.